Skip to content

New regression in python #69

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
hugtalbot merged 34 commits into from
Jan 15, 2026
Merged

New regression in python #69

hugtalbot merged 34 commits into from
Jan 15, 2026
+547 −0

Conversation

@epernod
Copy link
Contributor

@epernod epernod commented Aug 19, 2024 •
edited
Loading

This is a new version of Regression using SofaPython3. Feel free to improve it.
TODO:

  • Remove unnecessary test code on numpy export, as soon as we agree on the formart
  • Move all code of file/folder into a several files
    • For folder and scenes parsing
    • for code on numpy array comparison
  • Hide SOFA logs
  • Improve end report format
  • Add CSV format
  • Create branch with new reference files
    • Test it in a SOFA pr with a ci-depends-on
    • Compare reference to the old ones

Style based on .regression-test file but the last argument is the number of keyframe to test:
for example:
demos/mydemo.py 4000 1e-3 1 100
100 step will be tested with a threshold of 1e-3.
key frame will be: [0 40 80 ... 4000]*dt so 101 values.

Right now, at each key frame, full distance between the ref dofs and simulated dofs is computer.
totalError = sum(fullDist) accumulated over all key frames.
errorByDos = sum(fullDist)/nbrDofs accumulated over all key frames.

Computation is done using numpy array. Maybe better metrics could be used.

example of output:

Compare All sets: 100%|##########| 1/1 [06:30<00:00, 390.56s/it]
Scenes.regression-tests: 100%|##########| 6/6 [06:30<00:00, 46.54s/it].44it/s]
### Number of sets Done:  1
### Number of scenes Done:  6
### Number of scenes failed:  2
### Success: ../SofaLnRobotics/scenes\benchmarks/Bench_Heart_Judkins_Left.py | Number of key frames compared without error: 101
### Failed: ../SofaLnRobotics/scenes\benchmarks/Bench_Heart_Judkins_Right.py
    ### Total Error per MechanicalObject: [0.0010049364875422493, 0.0009830053353054394]
    ### Error by Dofs: [1.4074740721880243e-07, 3.991089465308321e-07]
### Success: ../SofaLnRobotics/scenes\benchmarks/Bench_Heart_SpireCatheter.py | Number of key frames compared without error: 101
### Success: ../SofaLnRobotics/scenes\benchmarks/Bench_inHeart_01_Judkins_Left.py | Number of key frames compared without error: 101
### Success: ../SofaLnRobotics/scenes\benchmarks/Bench_inHeart_01_Judkins_Right.py | Number of key frames compared without error: 101
### Failed: ../SofaLnRobotics/scenes\benchmarks/Bench_inHeart_01_SpireCatheter.py
    ### Total Error per MechanicalObject: [0.021162743293277383, 0.019589862858611747]
    ### Error by Dofs: [3.023249041896769e-06, 7.953659301100991e-06]

epernod and others added 24 commits December 24, 2024 13:13
@epernod @fredroy
Add new regression script fully in python
8bc76e7
@epernod @fredroy
Add structure to store Scenedata and parse regression-test files.
e3b9e30
@epernod @fredroy
start work on mechanicalState search
f6e7a17
@epernod @fredroy
Add scene ref path computation
dd2f779
@epernod @fredroy
Backup work on saving data
624e9ec
@epernod @fredroy
Update code to dump directly a map and add code to reload files. Not …
cd09082 
…working yet.
@epernod @fredroy
Fix python scene support in new_regression script
ea5e979
@epernod @fredroy
Fix export of numpy data
476fffc
@epernod @fredroy
Add file compression
9ce9190
@epernod @fredroy
backup work on comparison
97fd6cc
@epernod @fredroy
clean code
cd7f598
@epernod @fredroy
Fix period export
f36e015
@epernod @fredroy
Fix comparison pipeline and add number of tested scene
8c25afe
@epernod @fredroy
Fix compare mode and store the total error and errorByDofs
33c8ed2
@epernod @fredroy
Add method to print results at end and remove intermediate prints
6d546aa
@epernod @fredroy
Split new script into 3 files
c3cd349
@epernod @fredroy
backup work on replay mode
edae2f6
@fredroy
fixes and usability upgrades
8ca4bd3
@fredroy
massive renaming
4078d6f
@fredroy
add option to disable progress bars
b09dc4b
@fredroy
fix crash
e2811f1
@fredroy
add some safeguards
af52c40
@fredroy
fix writting references (check dir)
548e14d
@fredroy
fix printing list of np.float
c479684
@fredroy fredroy mentioned this pull request Jan 10, 2025
Open
hugtalbot
hugtalbot reviewed Jan 22, 2025
View reviewed changes
hugtalbot
hugtalbot reviewed Jan 22, 2025
View reviewed changes
@epernod
Rename RegressionSceneParsing.py into RegressionSceneList to match th…
8af8bbf 
…e class name. Rename internal variable to be more explicit on the type of object
@epernod
Copy link
Contributor Author

epernod commented Jan 13, 2026

New edits:
rename files:

  • SofaRegressionProgram.py -> main script to launch the regression
  • RegressionSceneList.py -> parse .regression-test file to list the scenes to tests and check if references are availables
  • RegressionSceneData -> parse reference files, run simulation and perform dof comparison

Add check and send error if MechanicalObject size is not the same between ref and current simulation

@epernod epernod mentioned this pull request Jan 14, 2026
Open
@hugtalbot hugtalbot merged commit 806c75e into sofa-framework:master Jan 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@hugtalbot hugtalbot hugtalbot left review comments

Assignees

No one assigned

Labels

enhancement pr: status ready

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@epernod @hugtalbot @fredroy